Export patch series: https://github.com/kakra/linux/pull/40.patch

btrfs: tiered allocation hints and queue-based read balancing

Special Thanks to @Forza-tng for extensive testing, feedback, and maintaining the documentation guide:
👉 Btrfs Allocator Hints and Read Policies Guide by Forza-tng

This PR introduces a set of patches to improve Btrfs performance and flexibility in heterogeneous storage environments (mixed SSD/HDD, tiered storage, bcache).

1. Allocator Hints (Data Placement)

Allows preferring specific devices for data or metadata allocations. This works by storing a hint in the persistent device item on-disk.

• Essential for tiered setups: Force metadata onto SSDs (for speed) while keeping bulk data on HDDs.
• Graceful removal: Mark devices to accept no new allocations, allowing to drain them naturally or via balance without racing against new writes.

2. Read Balancing Policies

Extends Btrfs RAID1 read balancing with a dynamic queue policy. The standard PID-based policy is often insufficient for mixed-device pools or high-IOPS workloads.

• pid: (Default) Static hashing by process ID. Good for simple setups.
• round-robin: Distributes reads equally. Good for aggregate throughput on identical disks.
• queue: (Recommended) Routes requests to the device with the fewest in-flight requests (shortest queue).
  • Adapts instantly to device load and speed differences.
  • Avoids "stalling" on busy devices.
  • In benchmarks, this policy consistently delivered the highest IOPS and lowest latency, especially under mixed load.
• devid: Pin reads to a specific device ID (mostly for testing).

(Note: Previous experimental latency-based policies were dropped in favor of queue due to better stability and lower complexity.)

3. Decoupling from Experimental Status

Important: Upstream Kernels (6.13+) use CONFIG_BTRFS_EXPERIMENTAL to gate various unstable work-in-progress features. To allow using the allocator hints and read policies without enabling potentially unstable upstream code, these features have been moved out of the experimental gate.

Recommendation: Remove the line CONFIG_BTRFS_EXPERIMENTAL from your .config before running make oldconfig. The build system will then prompt you specifically for the new options (CONFIG_BTRFS_ALLOCATOR_HINTS, CONFIG_BTRFS_READ_POLICIES), allowing you to enable them safely without turning on other experimental Btrfs features.

Quickstart Guide

Setting Allocator Hints

1. Enable CONFIG_BTRFS_ALLOCATOR_HINTS in kernel config.
2. Run btrfs device usage /mnt/path to identify your device IDs.
3. Set the hint via sysfs (this persists on-disk, no udev rules needed!):
   echo <TYPE> | sudo tee /sys/fs/btrfs/<UUID>/devinfo/<DEVID>/type

Available Types:

• 0: Prefer data (Default for HDDs).
• 1: Prefer metadata (Recommended for SSDs/NVMe).
• 2: Metadata only (Use with caution).
• 3: Data only (Use with caution).
• 4: None preferred (Avoids new allocations, useful to drain a drive).
• Added: 5: None (Strictly prevents ANY new allocation, useful for parallel device remove).

After changing hints, a rebalance of metadata/data is required to move existing extents to their preferred location.

Enabling Read Policy

1. Enable CONFIG_BTRFS_READ_POLICIES in kernel config.
2. Set boot parameter: btrfs.read_policy=queue
3. Or switch at runtime: echo queue | sudo tee /sys/fs/btrfs/<UUID>/read_policy

Diagnostic Statistics

Adds per-device read statistics to /sys/fs/btrfs/<UUID>/devinfo/<DEVID>/read_stats.

ios %lu wait %llu avg %llu age %llu ignored %llu

• ios: Total read I/O count.
• wait: Total accumulated wait time (ns).
• avg: Cumulative average read latency (ns).
• age: "Fairness" counter. Increments when the device is skipped/ignored during selection. Resets to 0 when selected. A constantly high age indicates the device is being avoided by the policy.
• ignored: Total count of times this device was a candidate but skipped.

Benchmark Results

The following benchmarks (based on kernel 6.12 LTS) compare the new policies against the defaults. Tests were performed on a mixed HDD RAID10 array with bcache, comparing an idle system vs. a system under heavy background load (defrag).

queue proved to be the superior all-rounder, effectively isolating foreground workloads from background noise.

Scenario: No Background Load

Scenario: Heavy Background Load (Defrag)

(latency-rr was an experimental hybrid policy used during testing, superseded by queue due to better performance and simplicity)

Changes in this version (Kernel 6.18 Port)

• Ported to Linux 6.18.
• Refactored Kconfig: Features are individually selectable and moved out of "Experimental".
• Robustness: Fixed potential NULL pointer dereferences in stats tracking during mount/unmount and race conditions in round-robin calculation.
• Simplified: Dropped complex EMA-based latency tracking in favor of the robust queue policy.

FAQ: Why is this not upstream?

1. Allocator Hints

The allocator hint patches (originally developed by Goffredo Baroncelli, now maintained here) have been discussed on the mailing list but were not merged for design reasons:

• Free Space Calculation (df): Btrfs calculates available space assuming any chunk can be allocated on any device (respecting RAID profiles). Restricting allocations via hints makes this calculation unreliable. Tools might report free space while Btrfs returns ENOSPC (No space left on device) because the allowed devices for a specific chunk type are full, even if other devices are empty.
• Maintenance: The original author ceased updates for newer kernels; this repository bridges that gap.

Compatibility Note: This patch reuses the existing (unused) type field in the device item on disk. It does not change the on-disk format version. Unpatched kernels simply ignore the value, ensuring data remains accessible (though allocation preferences will be lost until booted with a patched kernel).

2. Read Policies (queue)

The new queue policy is an experimental addition in this patch set. It is unlikely to be accepted upstream in its current form due to Layer Violation:

• The filesystem layer (Btrfs) directly accesses internal block-layer statistics (in-flight queue depth) to make routing decisions. The Linux kernel generally enforces strict separation between these subsystems.
• However, benchmarks show this cross-layer optimization yields significant performance gains in mixed setups, justifying its inclusion here.